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A flexible multimodal object tracking system. 

Accession number & update 

0007978290 20070101. 
Conference information 

Proceedings of International Conference on Image Processing, 
Barcelona, Spain, 14-17 Sept. 2003. 
Sponsor(s): IEEE Signal Process. Soc. 

Source 

Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429), 2003, vol.2, p. 

Ill— 1 33—6 vol.2, 6 refs, pp. 3 vol .(lxxii+1 138+1 126+1094), ISBN: 0-7803-7750-8. Publisher: IEEE, 

Piscataway, NJ, USA. 
Author(s) 

Breit-H, Rigoll-G. 
Author affiliation 

Breit, H., Rigoll, G., Inst, of Human-Machine-Commun., Munich Univ. of Technol., Germany. 
Abstract 

In this paper we present a flexible multimodal object tracking system. It is based on a particle filter, 
which combines the outputs of different measurement methods (also called modes or cues) in a flexible 
manner. The modes for locating the desired object can be selected depending on the specific object, 
which is to be tracked, and the environment of the object. By combining multiple modes we aim to add 
the strengths while at the same time to overcome the specific disadvantages of the different modes. Thus 
the robustness of the tracking system will be increased and a successful tracking will be possible in 
critical situations where a system using only a single mode would fail. We have used this approach for 
tracking persons in different environments. The measurement modes, which we have implemented for 
this purpose, are a pseudo 2-dimensional hidden Markov model (P2DHMM), a color based skin finder, 
and a motion detector. We describe the theory and the architecture of this tracking system and finally 
depict some exemplary results. 
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Classification codes 

B6135 Optical-image-and-video-signal-processing*; 
B61 40B Filtering-methods-in-signal-processing ; 
B0240J Markov-processes; 

C5260B Computer-vision-and-image-processing-techniques*; 
C1140J Markov-processes. 
Keywords 

multimodal-object-tracking; particle-filters; pseudo-2-dimensional- 
hidden-Markov-model; color-based-skin-finder; motion-detector; 
multiple-modes. 
Treatment codes 
P Practical; 

T Theoretical-or-mathematical. 
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Conference-paper. 
Availability 
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Publication year 
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Publication date 
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Proceedings 2001 International Conference on Image Processing, 
Thessaloniki, Greece, 7-10 Oct. 2001. 
Sponsor(s): IEEE Signal Process. Soc. 

Source 

Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205), 2001, vol.2, p. 

53-6 vol.2, 6 refs, pp. 3 vol.(lxx +1133+1108+1110), ISBN: 0-7803-6725-1. Publisher: IEEE, 
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Author(s) 

Breit-H, Rigoll-G. 
Author affiliation 

Breit, H., Rigoll, G., Dept. of Comput. Sci., Gerhard Mercator Univ., Duisburg, Germany. 
Abstract 

This paper presents the continuation of our work on object tracking in presence of non-stationary 
background using a combination of a pseudo-2D hidden Markov model (P2DHMM) and a Kalman filter. 
It presents a major improvement by introducing a novel method that allows an automatic adaptation of 
our system to the changing background. Other improvements of the system's tracking capabilities are 
achieved by refined person models and normalization procedures. One of the major goals of our 
approach to tracking is to achieve high quality tracking results despite non-stationary background that 
can be caused e.g. by moving objects in the background or by camera operations such as panning or 
zooming. In previous publications we demonstrated that our combined P2DHMM/Kalman filter approach 
is an interesting solution to this problem, because it enables us to perform person tracking without the 
use of motion information. In this paper, we show that this approach can be further improved by adapting 
the system to the constantly changing background. We further demonstrate that such a background 
adaptation is very difficult to achieve in standard tracking approaches but can be effectively realized in 
our combined P2DHMM/Kalman filter approach. The effectiveness of this new procedure is demonstrated 
in experiments, where the tracking results and the quality of the person segmentation of our original 
system is compared to the results obtained with the improved approach. 
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